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1.  INTRODUCTION 


V.'c  shall  give  a  finitely  convergent  procedure  for  solving  the  problem 
(P)  Maximize  f(x)  subject  to  ax  +  b  ^  0,  i  =  1,  .  .  .  ,  m, 

X  1  1 

where  f  is  a  strictly  concave  and  differentiable  function  of  the  n-vector 
x  which  assumes  its  unconstrained  maximum.  The  a^  are  n-vectors  and 
the  b^  are  scalars. 

No  additional  assumptions  whatever,  for  example  regarding  the 
exclusion  of  degeneracy  or  the  existence  of  a  feasible  solution  of  (P)  , 
are  required.  In  fact  the  presence  of  degeneracy  can  only  hasten  conver¬ 
gence,  and  no  perturbation,  either  real  or  virtual,  is  ever  required. 

If  (P)  is  infeasible,  the  procedure  detects  this  fact  routinely  and 
terminates  with  a  display  of  an  infeasible  subset  of  the  constraints. 

Thus  no  feasible  solution  is  needed  to  initiate  the  calculations. 

An  interesting  aspect  is  that  provision  is  made  for  making  effective 
use  of  prior  information  which  may  be  available  regarding  which  constraints 
are  likely  to  be  binding  at  the  optimum  solution  of  (P)  (under  the  tenta¬ 
tive  assumption  that  (P)  is  feasible) . 

The  present  method  is  tot  a  "gradient"  or  "simplicial"  or  approximate 
method.  It  is  aimed  directly  at  constructing  a  solution  of  a  certain 
version  of  the  Kuhn-Tucker  Conditions  [7]  for  (P) ,  which  are  necessary 
and  sufficient  for  an  optimal  solution.  It  can  be  interpreted  as 
solving  a  finitely  convergent  and  "slowly  changing"  sequence  of  simpler 
maximization  problems  involving  only  linear  equality  constraints. 

When  f(x)  is  quadratic  the  method  specializes  to  a  promising 
generalization  of  Theil  and  van  de  Panne's  algorithm  for  quadratic 


programming  [8],  with  the  added  generality  being  that  (a)  certain 
restrictive  assumptions  (regarding  the  feasibility  of  (P)  and  the 
exclusion  of  degeneracy)  are  shown  to  be  completely  unnecessary,  and  (b) 
provision  is  made  for  utilizing  prior  information  regarding  a  given  problem 
so  as  to  shorten  the  calculation  time. 

In  order  not  to  obscure  the  fundamental  approach,  in  the  next  section 
we  derive  the  basic  procedure  and  illustrate  it  by  example  under  two 
simplifying  assumptions.  These  assumptions  are  removed  in  the  following 
section.  Specialization  to  the  quadratic  case  is  the  subject  of  the 
penultimate  section,  and  finally  we  discuss  some  computational  considerations 
and  give  a  modification  which  permits  linear  equality  constraints  to  be 
handled  efficiently. 

Lengthy  Interruptions  of  the  text  are  avoided  by  placing  formal 
statements  and  proofs  of  all  theorems  in  the  Appendix  . 

By  way  of  preliminaries,  we  introduce  two  definitions.  A  subset  S 
of  constraint  indices  is  said  to  be  cons  is  tent  when  the  set  (x:  a^x 
+  b^  =  0,  i  e  s)  is  not  empty,  and  independent  when  the  n-vectors  a^, 
i  e  S,  are  linearly  independent.  7ho  empty  sot  is  considered  to  be  both 
consistent  and  independent.  We  denote  by  M  the  set  of  the  first  m  posi¬ 
tive  integers,  where  m  is  the  number  of  constraints  of  (P)  .  The  gradient 
of  f(x)  is  denoted  by  ^f(x). 
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2.  DEVELOPMENT  UNDER  T'.VO  SIMPLIFYING  ASSUMPTIONS 

Throughout  this  paper,  f(x)  is  assumed  to  be  as  stated  in  the 
introduction.  For  the  sake  of  expository  convenience,  in  this  section  we 

also  temporarily  assume  (a)  that  (P)  is  feasible,  which  implies  that  (P) 

$ 

has  a  unique  optimal  solution  x  ,  and  (b)  that  the  set  B  =  {i  e  M:  a.x  +  b^=  0 ] 
is  independent. 

Since  our  computational  procedure  is  based  on  constructing  a  solution 
of  the  Kuhn-Tucker  Conditions  for  (P)  ,  we  begin  by  recording  without 
proof  a  version  of  Theorem  3  from  [7],  as  specialized  to  (P)  .  This 
particular  version  was  introduced  in  [6,  Ch.  3 J . 

Theorem  (Kuhn-Tucker)  : 

Assume  that  f(x)  is  differentiable  and  concave.  Then  a  trial  solution 
x°  is  an  optimal  solution  of  (P)  if  and  only  if  for  some  subset  S°  c  M 
of  constraint  indices  there  exist  m  real  numbers  (dual  variables)  u° 
such  that  (x°,  u°)  satisfies  the  associated  (Kuhn-Tucker)  conditions: 

f'(KT-l)  *  f(x)  +  S  u  a  =  0 
(=S)  ics  1  1 

(^(KT-2)  a^x  +bi  =  0,  i  e  S;  u^  =  0,  ie  M-S 

(KT-3)  aix  +  b  *  0,  i  e  M-S 

(KT-4)  u  *  0,  i  e  S  . 

The  method  suggested  below  for  constructing  a  solution  of  the  Kuhn- 
Tucker  Conditions  (KT-1,...,  4)  involves  solving  for  certain  trial  sets 
S  C  m  ,  the  equations  (KT-1)  and  (KT-2)  (together  designated  by  the  cymool 
(=S)  )  and  then  checking  the  inequations  (KT-3)  ana  (KT-4)  to  see  if  they 
are  satisfied.  If  they  are  satisfied,  the  desired  optimal  solution  of  (P) 
is  obviously  at  hand..  This  approach 


suggests  the  following  important  definition.  A  set  S°  c  m  is  said  to 
be  valid  if  and  only  if  there  exists  a  pair  (x°,  u°)  which  satisfies 
(KT-1,  .  .  .  ,  4)  associated  with  S°  .  It  is  shown  in  Proposition  1 
(see  the  Appendix  for  the  proofs  of  this  and  subsequent  propositions) 

that  for  any  consistent  and  independent  trial  set  S  C  m  the  equations 

s  s 

(=S)  have  a  unique  solution  (x  ,  u^)  ,  and  it  will  be 
shown  in  see  ticn  2  J.  bcl  ov  that  there  is  at  least  one  valid  set, 
and  that  every  valid  set  is  consistent  and  independent.  Thus  we  see  that 
the  problem  of  solving  (P)  can  be  viewed  as  the  problem  of  finding  a 
valid  set  S  ;  for  then  by  the  definition  of  validity  (=S*)  yields 

as  its  unique  solution  an  ootimal  solution  x*  of  (P)  and  the  corres- 

* 

ponding  dual  variables  u  .  How  one  may  go  about  finding  a  valid  set 
is  the  main  concern  of  the  balance  of  this  paper. 

2 . 1  Characterizing  a  Valid  Set 

It  is  shown  in  Proposition  2  that  there  exists  at  least  one  triple 

♦  ♦  *  *  ♦ 

(x  ,  u  ,  S  )  satisfying  (KT-1,  .  .  .  ,  4)  ,  and  that  (x  ,  u  )  is 

unique.  It  is  clear  from  the  nature  of  (KT-1,  .  .  .  ,  4)  that  S* 
can  be  taken  as  any  set  S  satisfying  A  C  s  c  b  ,  where  we  make  the 
def initions 

A  =  {i  e  M:  u^  >  0} 

B  =  (i  e  M:  a^x  +  b^  *  0}  . 

In  words,  the  set  A  is  the  collection  of  constraint  indices  corresponding 
to  the  constraints  which  are  active  at  the  optimum  of  (P)  in  the  sense 
that  their  dual  variables  are  strictly  positive,  and  B  is  the  set  of 
indices  corresponding  to  constraints  which  are  binding  at  the  optimum. 

* 

From  (KT-2)  and  (KT-4)  ,  A  C  B  .  Conversely,  it  is  clear  that  S 
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must  satisfy  A  c  s  C  b  ,  or  else  (x  ,  u  ,  S)  cannot  satisfy 
(KT-1,  .  .  .  ,  4)  . 

From  these  and  previous  remarks  we  have  the  following  characterizations 
of  validity:  S  c  m  is  valid  if  and  only  if  A  c  s  c  b  if  and  only  if 

s  s 

S  is  consistent  and  independent  and  (x  ,  u  )  satisfies  (KT-3)  and 
(KT-4)  in  addition  to  (=S)  . 

Unfortunately,  neither  A  nor  B  are  likely  to  be  known  a  priori, 
so  that  the  identity  of  a  valid  set  is  not  immediately  available  from 
the  first  characterization.  From  the  second  characterization  we  do  have, 
however,  (a)  the  useful  necessary  condition  that  a  valid  set  must  be 
consistent  and  independent,  which  enables  the  search  to  be  restricted  to 
such  sets,  and  (b)  a  convenient  test  for  validity  to  apply  to  such 
candidates . 

2.2  Determining  the  Order  of  Trials 

The  fact  that  there  is  but  a  finite  number  of  subsets  of  M 

immediately  establishes  the  existence  of  a  finite  procedure  for  finding  a 

valid  set--enumeration  of  consistent  and  independent  sets.  A  non-repeating 

sequence  <SU>  of  such  trial  sets  terminates  at  the  first  trial  for  which 
u  v 

(xS  ,  u  )  satisfies  (KT-3)  and  (KT-4)  associated  with  SU  ,  i.e. 

u 

at  the  first  valid  S  .  When  m  is  large,  of  course,  undirected  enumer¬ 
ation  is  likely  to  be  computationally  impractical.  We  therefore  develop 
rules  for  directing  the  enumeration  so  as  to  keep  the  number  of  necessary 
trials  relatively  small. 

If  S  is  net  valid,  then  either  S-B  /  or  A  -S  ^  0  ,  or  both. 

For  obvious  reasons  the  set  S-B  will  be  called  the  excess  of  S  , 
and  A  -S  will  be  called  the  deficiency  of  S  .  Clearly  the  smallest 


change  in  S  which  would  lead  to  a  valid  set  would  be  to  add  its  deficiency 
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and  delete  Its  excess.  The  number  of  indices  in  {A  -S}U(S-B  }  is 

therefore  a  measure  of  the  distance^  d(S)  between  S  and  the  nearest 

valid  set.  If  S  is  consistent  and  independent,  as  well  as  invalid, 

S 

then  by  the  second  characterization  of  validity  we  have  a^x  +  b^  <  0 

S 

for  seme  i  e  M-S  ,  or  u^  <  0  for  some  i  e  S  ,  or  both.  An  event  of 
the  former  type  will  be  called  a  feasibility  alarm,  and  an  event  of  the 
latter  type  an  optimality  alarm.  A  feasibility  alarm  from  the  deficiency 
will  be  called  a  real  alarm,  as  will  an  optimality  alarm  from  the  excess 
of  S  . 

An  appealing  conjecture  is  that  the  feasibility  alarms  coincide  with 
the  deficiency,  and  that  the  optimality  alarms  coincide  with  the  excess  of 
a  consistent  and  independent  but  invalid  set  S  .  If  this  were  true, 
then  by  solving  (=S)  one  could  immediately  determine  the  identity  of  a 
valid  set.  Unfortunately  this  conjecture  of  perfect  coincidence  is  not 
true  in  general,  as  can  easily  be  shown  by  counterexample  (even  for  the 
simple  case  n  =  2  and  f(x)  diagonal  quadratic;  see  the  example  of 
section  2.3).  What  can  be  shown  (Proposition  3),  however,  is  that  at  least 
one  alarm  is  real.  This  result  suggests  a  procedure  for  determining  the 
order  of  trials  by  essentially  heeding  the  alarms  one  at  a  time.  By 
heeding  an  alarm  we  mean  adding  to  S  a  constraint  which  gives  a  feasibility 
alarm,  or  deleting  from  S  a  constraint  which  gives  an  optimality  alarm. 

For  convenience  of  exposition,  we  assume  temporarily  that  is  consistent 

/ 

-  The  distance  beiveen  a  set  C  C  M  and  a  set  D  C  M  can  be 
defined  as  the  nuvber  cf  elements  in.  the  symmetric  difference  set 


^C-D  )lj  (D-C }  . 
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and  independent  whenever  S  is  consistent  and  independent  and  i  is  a 

feasibility  alarm  due  to  S  ;  this  assumption  will  then  be  removed. 

Let  S°  be  an  arbitrary  consistent  and  independent  initial  trial  set, 

and  let  it  be  invalid  (i.e.  let  d(S°)  ^  1  ).  If  S°  happens  to  yiold 

exactly  one  alarm  i  ,  then  by  Proposition  3  it  is  real  and  therefore 

o 

d(S°  ±  i  )  =  d(S°)  -  1  .  Here  we  use  the  notation  S°  i  i  to  mean 
o  o 

S°Ui  or  S°-i  according  as  i  f!  S  or  i  e  S  (i.e.  according  as 
o  o  o  o 

i  is  a  feasibility  or  an  optimality  alarm),  in  order  to  avoid  having  to 
o 

distinguish  between  feasibility  and  optimality  alarms.  If  S°  yields 

more  than  one  alarm,  then  by  Proposition  3  we  have  d(S°  ±  i)  «  d(S°)  -  1 

o 

for  some  i  e  T  ,  where  T  is  the  collection  of  alarms  due  to  S 

o 

Therefore  by  trying  each  of  the  sets  S  £  1  ,  i  e  T  ,  we  would  obtain 

at  least  one  set  which  is  closer  to  the  closest  valid  set  (and,  in  fact, 

closer  to  every  valid  set).  We  call  this  the  first  generation  of  trials. 

If  d(S°)  =  1  ,  then  a  valid  set  is  found  during  the  first  generation  of 
o 

trials;  if  d(S  )  >  1  ,  a  second  generation  of  trials  is  necessary. 

For  each  1CT  ,  define  T  to  be  the  set  of  alarms  generated  by 

S°  ±  i  .  Since  d(S°)  >  1  ,  we  have  T  ^  0  (i  e  T)  .  The  second  gener- 

o 

ation  of  trials  consists  of  the  sets  S  ±  i  r  j  for  all  i  e  T  and 
j  e  T  .  The  symbol  S°  ±  i  ±  J  means,  of  course,  (S°  ±  i)Uj  if 
j  t  (S°  ±  i)  and  (S°  ±  i)  -  j  if  j  e(S°  ±  i)  .  By  Proposition  3, 
d(S°  ±  i  ±  J)  =  d(S°)  -  2  for  some  i  e  T  and  j  e  t  .  If  d(S°)  =  2  , 
then  a  valid  set  is  found  at  this  generation;  if  d(S°)  >  2  ,  a  third 
generation  is  necessary. 

Continuing  analagously,  one  constructs  third  and  higher  order 
generations  as  necessary.  If  at  any  trial  a  set  is  encountered  which  has 
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been  tried  before,  it  may,  of  course,  be  discarded. 

It  is  clear  that  this  search  strategy  terminates  after  exactly  d(S°) 
generations  of  trials  with  the  (unique)  valid  set  which  is  closest  to  S°  , 
and  that  computational  economies  can  be  achieved  due  to  the  fact  that 
changes  in  trial  sets  involve  adding  or  deleting  only  one  constraint 
at  a  time.  A  summary  follows. 


Rules  for  Ordering  the  Trials 

0.  Let  S°  be  conelctont  and  into  pendent,  and  let  T  be  the  set  of 
alarms  determined  by  S°  .  If  T  =  fb  ,  then  S°  is  valid  and  the 
procedure  terminates;  otherwise,  a  first  generation  of  trials  is 
necessary . 

1.  At  the  first  generation  of  trials,  for  each  i  e  T  let  T 

denote  the  set  of  alarms  generated  by  S°  ±  i  .  If  =  0  for 

*  o  * 

some  i  e  T  ,  then  S  ±  i  is  valid  and  the  procedure  terminates; 
otherwise,  a  second  generation  of  trials  is  necessary. 

2.  At  the  second  generation  of  trials,  for  each  i  t  T  and  J  e  , 

let  T  be  the  set  of  alarms  generated  by  b°  ±  i  :  j  ,  If 

•  J 

.  *  *  o  *  * 

T  o  J0  for  some  i  e  T  ,  and  j  e  T^+  ,  then  S  ±  i  J 

is  valid  and  the  procedure  terminates;  otherwise,  a  third  generation 
of  trials  is  necessary. 

3.  Third  and  higher  order  generations  of  trials  are  constructed 
analagously . 


(if  at  any  trial  a  set  is  encountered  which  has  been  tried  previously, 


it  may  be  discarded). 


It  has  been  temporarily  assumed  that  SUiQ  is  consistent  and  independent 
whenever  S  is  and  i^  is  a  feasibility  alarm  due  to  S  .  By  the  nature 
of  the  above  rules,  the  effect  of  this  assumption  is  to  ensure  that  every 
trial  set  at  every  generation  is  consistent  and  independent,  so  that 
Proposition  3  can  always  be  applied.  This  assumption  may  be  dropped  by 
appealing  instead  to  Proposition  4,  which  asserts  that  SiJi^  Is  independent 
when  it  is  consistent,  and  that  if  SUi  is  inconsistent  then  the  equations 


,  E  z  a  +  a , 

ieS  i  i  i 


0 


have  a  unique  solution  z  ,  z^  <  0  for  at  least  one  i  in  the  excess 

of  SUi  ,  and  {(SUi  )  -  i}  is  consistent  and  independent  for  any  i 
o  o 

<5 

such  that  z  ^  <0  .  It  follows  that  the  above  rules  are  applicable 

if  we  interpret  "uptinolity  alarms"  *e  being  defined  for  inconsistent  sets  S!Ji 

S 

as  the  constraints  corresponding  to  <  0  .  The  resulting  procedure  is 
given  in  Figure  1  (ignore  for  the  time  being  the  positive  branch  of  step 
2b,  which  is  irrelevant  when  (P)  is  feasible). 

Step  4  of  Figure  1  can  obviously  oe  omitted  when  an  optimality  alarm 
is  oeing  heeded,  for  the  result  can  only  be  another  consistent 
ana  independent  set.  It  can  also  be  omitted  when  a  feasibility  alarm  is 
being  heeded.  For  example,  when  f(x)  is  quadratic  (see  section  4) 
step  3  can  go  directly  to  step  la  and  any  inconsistency  will  be  auto¬ 
matically  detected  during  the  calculations,  thereby  commanding  transfer 
of  control  to  step  lb.  When  a  method  is  us?d  for  step  la  which  does  not 
routinely  and  quickly  reveal  inconsistency  of  the  trial  set,  step  3  should 
go  to  step  lb  when  a  feasibility  alarm  is  being  heeded — if  the  equations  of 
step  lb  have  no  solution  (and  efficient  matrix  partitioning  methods 
[4;  Appendix  C  of  6]  should  be  used  here)  then  control 
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Choose  an  initial 
consistentoand  inde¬ 
pendent  S  ,  and  put 


la 


I 

* 


lb 


Solve  the  system  of 
equations 

Vf(x)  +  .E  u.  a  =  0 
ieS  i  i 

a.x+bJ=0,ieS 
1  i  * 

u.  =  0  ,  i  g  M  -  S 

l 

for  its  (unique) 

S  5 

solution  (x  ,  u  ) 


2a 


Solve  the  system  of 
equations 


E  z  a .  +  a  =0 
i  i  i 

S-i  0 

o 

for  their  (unique)  solution 


z  ,  where  i  is  the  index 
o 

of  the  feasibility  alarm 
which  led  to  S 


2b. 


Yes; 


u  2:  0  ,  V  i  e  S 


\  No  _iIo 


i 


\  a.x  +  b.  i  0  ,  v  i  6  H  •  S  t  ' 

'  * _  / 


•  m  ,  /  S  S\  /  ^  ^  \ 

Terminate;  (x  ,  u  )=(x  ,  u  ) 


3. 


\  *  0  ,  V  i  e  c- 


\Yes 


u  —  i  ?  » 

o  ' 


Terminate;  (p)  infeasible/* 


Choose  the  next  S  in 
accordance  with  the  rules 
of  section  2.2 


Yes 


S  consistent? 


)  _NQ. 


Figure  1 


A  Procedure  for  Concave  Programming  With  Linear  Constraints 

*  i  , 

As  noted  in  the  text,  step  4  can  and  should  be  incorporated  (at  little  or 
no  additional  computational  cost)  into  steps  la  or  2b. 
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must  be  transferred  to  step  la. 

o 

Remark:  If  the  initial  trial  set  S  can  be  chosen  to  be  a  subset  of 

B  ,  (e.g.  S°  =  0  )  then  all  optimality  alarms  can  be  ignored,  as  can  any 

feasibility  alarm  which  leads  to  an  inconsistent  set  when  heeded.  This 

follows  because  any  consistent  and  independent  but  invalid  set  with  no 

excess  giv.s  at  least  one  (feasibility)  alarm  from  its  deficiency,  and 

heeding  that  alarm  must  obviously  lead  to  another  consistent  and  indo- 

o 

penoent  sot.  Similarly,  If  S  can  be  chosen  to  contain  A  ,  then  all 
feasibility  alarms  can  be  ignored.  Thus  certain  types  of  prior  information 
lead  to  considerable  simplifications. 

2 . 3  An  Example 

The  example  of  Figure  2  is  designed  to  illustrate  the  operation  of 
the  algorithm  and  show  that  not  all  alarms  need  be  real.  For  convenience 
we  present  a  graphical,  rather  than  numerical,  example  in  two  dimensions 
(n=2)  .  We  take  f(x)  to  be  the  ordinary  distance  from  x  to  a  fixed 
point  xq  .  By  the  Kuhn-Tucker  Theorem,  (=S)  is  a  necessary  and 
sufficient  condition  for  a  maximum  of  f(x)  over  all  x  in  the  linear 

manifold  (x:  a^x  +  =  0,  i  e  s)  .  Hence  (=S)  yields  as  its  solution 

S 

the  orthogonal  projection  x  of  x^  onto  the  manifold  and  the 

s  s 

coefficients  u^  of  f(x  )  expressed  in  terms  of  the  gradients 
a^  ,  i  e  S  .  Loci  of  the  four  constraints  are  drawn  and  labeled  with 

constraint  indices;  their  gradients  are  also  drawn  in  at  selected  points. 

* 

The  feasible  region  is  hatched  and  x  is  narked  by  a  heavy  dot. 

Let  the  initial  consistent  and  independent  trial  set  be  arbitrarily 
chosen  as,  say,  S°  =  (3,4)  .  S°  is  easily  seen  to  yield  a  feasibility 
alarm  for  i  =  1  and  optimality  alarms  for  i  =  3  and  4  .  Hence  the 


Figure  2 
An  Example 
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first  generation  trial  sets  are  (3,4,1}  ,  (4)  ,  and  {3}  .  Now  (4) 
yields  only  ono  alarm,  an  optimality  alarm  fcr  i  e  4  ;  {3}  also  yields 
one  alarm,  a  feasibility  alarm  for  i  =  1  ;  and  (3,4,1)  is  inconsistent 
solving  z  a  +  zAaA  +  a,  =  0  ,  one  obtains  optimality  alarms  for  both 
i  =  3  and  4  .  Hence  the  second  generation  trial  sets  are  0  ;  (3,1)  ; 
(4,1}  and  (3,l)  .  V/e  find  that  0  yields  feasibility  alarms  for 
i  =  1,2,  and  3  ;  that  (4,l}yields  optimality  alarms  for  i  =  1  and  4 
and  that  (1,3}  yields  no  alarms  at  all.  Hence  (1,3)  is  valid  and  the 
computations  terminate.  A  diagrammatic  summary  of  the  trials  is  given  in 
Figure  3.  Of  course,  A  =  B  =  (1,3}  is  obvious  by  inspection  of  Figure  2 
so  that  a  valid  set  has  indeed  been  found.  Note  that  d((3,4})  =  2  ,  and 
that  a  valid  set  was  indeed  found  in  two  generations. 

Looking  back  over  the  calculations,  we  observe  that  (3,4}  gave 
rise  to  a  optimality  alarm  for  i  =  3  which  was  not  real;  and  that  0 
gave  rise  to  a  feasibility  alarm  for  i  =  2  which  was  not  real.  Hence 
not  all  alarms  are  real. 


Zeroth  Generation 
First  Generation 
Second  Generation 


A  Summary  of  a  Sequence  of  Trials 


3.  REMOVING  THE  SIMPLIFYING  ASSUMPTIONS 


It  is  now  shown  that  the  assumptions  regarding  the  feasibility  of 
(p)  and  the  independence  of  B  can  be  dropped  without  impairing  the 
effectiveness  of  the  procedure  presented  in  the  previous  section. 

3.1  Dropping  the  Assumption  that  B  is  Independent 

* 

If  B  is  dependent,  u  need  not  be  unique.  Thus  A  need  not 
be  uniquely  defined,  and  although  the  definition  of  a  valid  set  remains 
as  before  we  no  longer  have  the  characterization  A  c  S  c  B  of  validity. 
In  place  of  A  we  define  the  collection  A^  (j  =  1, . ..,  k)  of  all 
minimal  valid  sets,  where  a  ralid  set  is  said  to  be  minimal  when  no 
proper  subset  of  itself  is  valid.  It  is  shown  in  Proposition  5  that  there 
is  at  least  one  minimal  valid  set  and  that  each  one  is  consistent,  inde- 
pendent,  a  subset  of  B  ,  and  u^  >  0  ,  i  c  A^  .  It  is  not  difficult  to 
see  from  the  nature  of  the  Kuhn-Tucker  Conditions  that  a  characterization 
of  validity  is  A^  c  S  c  B  for  some  1  s  J  £  k  .  If  S  is  consistent 

s  s 

and  independent,  it  is  obvious  that  S  is  valid  if  and  only  if  (x  ,  u  ) 
satisfies  (KT-3)  and  (KT-U)  . 

The  procedure  of  Figure  1  rests  on  Propositions  1  through  4. 
Propositions  1  and  U  and  the  first  part  of  Proposition  2  do  not  require 
B  to  be  independent.  Proposition  5 »  as  we  have  Just  shown,  takes  the 
place  of  the  second  part  of  Proposition  2.  Proposition  6  is  designed  to 
take  the  place  of  Proposition  3*  It  follows  that  the  procedure  of 
Figure  1  still  applies,  and  that  at  each  generation  of  trials  at  least 
one  trial  set  is  one  unit  of  distance  closer  to  the  collection  of  all 
valid  sets  than  any  trial  set  at  the  previous  generation. 
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t  However,®  exceptional  cose  arises  when  a  trial  set  S  is  not  valid  but 

S  * 

nevertheless  x  =  x  ,  a  possibility  that  can  arise  only  when  B  is 

dependent  (an  example  of  this  situation  is  given  in  the  Appendix).  In 

Proposition  6  it  is  shown  that  in  this  situation  at  least  one  optimality 
k  /  \ 

alarm  is  from  (S  -  A^)  ,  so  that  by  heeding  the  alarms  one  at  a  time 

S  will  lead  to  at  least  one  set  which  is  closer  to  some  A  .  Thus 

J 

convergence  is  still  assured. 


3.2  Dropping  the  Assumption  that  J>L.  is  feasible 

Assume  that  (p)  has  no  feasible  solution.  We  shall  show  that 
the  procedure  of  Figure  1  is  well-defined  and  terminates  in  the  positive 
branch  of  step  2b  at  the  first  trial  set  encountered  which  corresponds 
to  an  infeasible  subset  of  the  constraints. 

The  initial  trial  set  can  be  chosen  to  be  the  empty  set,  any 
singleton,  or  any  other  consistent  and  independent  set  as  usual.  By 

s  s 

Proposition  1  the  equations  (=S)  hove  a  unique  solution  (x  ,  u  )  so 
long  as  S  is  consistent  and  independent,  and  clearly  every  such  set 
leads  to  at  least  one  feasibility  alarm  (otherwise  (p)  would  be 
feasible).  Furthermore,  the  first  part  of  Proposition  4  still  holds. 

It  follows  that  the  procedure  of  Figure  1  is  well-defined  and  cannot 
terminate  in  the  positive  branch  of  step  2a.  It  remains  to  show  that 
the  procedure  does  terminate  in  the  positive  branch  of  step  2b. 
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A  set  S  c  M  of  constraint  indices  is  said  to  be  infeasible  when 

{x:  aix  +  bi  0  ,  i  e  S}  is  mpty,  and  is  said  to  be  minimally  infeasible 

when  no  proper  subset  of  itself  is  infeasible.  Let  D,(j  =  1, . d)  be 

J 

the  collection  of  distinct  minimally  infeasible  sets.  Then  clearly  ScH 

is  infeasible  if  and  only  if  DCS  for  some  1  s  j  <d  ,  and  consis- 

J  ™ 

tency  implies  feasibility  but  not  conversely. 

Every  consistent  end  independent  trial  set  S  yields  at  least  one  feasi¬ 
bility  alarm  from  every  D  -  S  ,  for  otherwise  some  D  would  be  feasible. 

J  J 

Proposition  7  asserts  that  every  inconsistent  and  feasible  trial  set 
which  can  arise  during  the  procedure  of  Figure  1  yields  at  least  one 
"optimality  alarm"  at  step  2b  from  ^6^  (S  -  D^)  .  Thus  at  each  generation 
of  trials  at  least  one  trial  set  is  one  unit  of  distance  closer  to 
the  collection  of  all  infeasible  sets  than  any  trial  set  at  the  previous 
generation,  except  for  certain  situations  (associated  with  inconsistent 
and  feasible  trial  sets)  when  one  can  assert  only  that  at  least  one 
trial  set  is  one  unit  of  distance  closer  to  at  least  one  of  the  mini¬ 
mally  infeasible  sets.  The  occurrence  of  an  inconsistent  and  infeasible 
trial  set  in  a  finite  number  of  generations  is  thereby  assured.  A 
crude  upper  bound  on  the  required  number  of  generations  is  the  distance 
from  the  initial  trial  set  to  the  minimally  infeasible  set  which  is 

g 

farthest  from  it.  Proposition  7  also  asserts  that  z  ^  0  for  all 
infeasible  sets  which  arise  in  the  procedure  of  Figure  1.  Thus  termination 
in  the  positive  branch  of  step  2b  is  assured. 

Remark:  Hie  remark  at  the  end  of  section  2.2  can  be  generalized  to  the 
case  where  it  is  not  known  whether  or  not  (p)  is  feasible.  Let  the 
initial  trial  set  be  the  empty  set  and  ignore  all  optimality  alorms  as 
well  as  feasibility  alarms  which  lead  to  an  '  .» 


sistent  set  when  heeded 
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This  strategy  was  seen  to  be  justified  when  (p)  is  feasible.  When  (p) 
is  infeasible,  since  trial  sets  can  only  grov  as  generations  pass  the 
situation  will  eventually  be  reached  where  a  next  generation  cannot  be 
defined  because  every  feasibility  alarm  leads  to  an  infeasible,  and 
therefore  inconsistent,  set.  This  situation  is  the  signal  for  termination 
when  this  strategy  is  used. 


b.  THE  QUADRATIC  CASE 

Specialization  to  quadratic  programming  is  natural  and  easy. 

When  f(x)  =  ^x^Cx  +  cSc  ,  where  C  is  an  n  by  n  negative 
definite  (symmetric)  matrix  and  c  is  an  n-vector,  f  meets  the 
requirements  stated  in  the  introduction  and  (=S)  is  a  system  of  linear 
equations  with  the  unique  solution  (assuming  that  S  is  consistent  and 
independent^/ ) 


US  =  CAS  C_1  As  r1^  ’  As  C"lc)  ;  \  =  0  ,  i  e  M-S 

S  _-l,  .t  Sv 
x  =  -C  (c  +  ag  ug)  , 


where  t  superscripts  denote  transpose,  A^  is  a  matrix  with  rows  a.  , 

O  X 

5 

i  e  S  ,  and  uc  and  b  are  similarly  defined.  A  computer  routine  for 

O  O 

implementing  Figure  1  would  invert  C  at  the  cutset,  along  with 

(A  o  C  ^  A^«)  ,  and  thereafter  use  efficient  partitioning  methods  (see, 

O  O 

e.g.,  [^]  and  Appendix  C  of  [6])  for  recomputing  (A  C-1  A^)'1  as  S 

O  O 

is  altered  by  one  constraint  at  a  time.  A  similar  strategy  applies  to 

the  solution  of  .  z,a.  +  aj  =  0  ,  which  con  be  written  (and  this 

ieS-i  i  i  i  *  ' 

o  o 

formula  applies  whether  or  not  f  is  quadratic) 


S  were  dependent, 


- 1  At,-1 

C  A  ] 


would  fail  to  exist. 


.18 


=  -a 


S-i 


(A 


_  .  a;  . 

S-i  S-i 


-1 


0  0  0  0 

Additional  computational  economies  by  partitioning  can  be  obtained  when 
some  of  the  constraints  are  simple  non-negativity  constraints  on  the 
variables . 


4,1  Relation  to  the  Algorithm  of  Theil  and  van  de  Panne 

The  oresent  algorithm  specializes  to  Theil  and  van  de  Panne's 
algorithm  [8]  for  quadratic  programming  under  their  assumptions^/ 

[8,  p.  9L  which  can  be  phrased  as  follows:  (a)  (p)  has  at  least 

two  different  feasible  solutions,  (b)  C  is  negative  definite,  (c) 

O 

a^.x°  +  b^  /  0  ,  i  c  M-S  ,  for  every  consistent  set  S  ,  and  (d) 
the  initial  trial  set  is  the  empty  set.  Assumption  (c)  is  strong  and  often 
difficult  to  verify.  It  can  be  shown  to  be  equivalent  to  requiring  for 
every  consistent  set  3  that  if  (x  ,  u  )  satisfies  (=S)  then 

g 

u^  /  0  for  all  i  c  3  ;  in  other  words,  degeneracy  is  completely 
excluded.  Assumption  (d)  is  not,  of  course,  an  assumption  in  the  true 
sense  of  the  word,  but  rather  a  convention- -when  it  is  followed,  the 
remark  at  the  end  of  section  2.2  applies  and  the  procedure  af  Figure  1 
is  simplified. 

The  developments  of  this  paper  show  that  Theil  and  van  de  Fame's 
algorithm  is  not  impaired  if  their  assumption  (c)  is  dropped,  and  that 
assumption  (a)  may  also  be  dropped  if  it  ic  understood  that  the  infeasibility 
of  (p)  is  distinguished  by  the  occurrence  of  a  generation  at  which  at 
least  one  feasibility  alarm  occurs  and  every  feasibility  alarm  leads  to  an 


o  / 

-'Boot  rl,2,3]  has  rederived  the  algorithm  of  Theil  and  van  de  Fanne, 


but  invokes  similar  assumptions  (although  he  weakens  assumption  (c)  somewhat). 
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inconsistent  set.  An  important  extension  is  obtained  if  assumption 
(d)  is  dropped  i/hen  a  priori  information  regarding  B  is  available,  for 
then  S°  can  very  likely  be  chosen  such  that  d(S°)  <  d(0)  (hopefully 
d(S°)  «  d(0)  )  and  the  calculation  time  can  thereby  be  greatly  reduced. 

5.  Discussion 

It  is  evident  that  the  algorithm  of  Figure  1  Trill  be  efficient 
when:  (a)  only  a  short  time  is  required,  on  the  average,  to  solve  ( =S • ) 
when  (=S'*)  has  been  solved  and  S"  differs  by  only  one  index  from  S*  ; 
and  (b)  S°  can  be  chosen  so  that  d(S°)  is  reasonably  small,  as  it 
often  can  be  when  a  priori  knowledge  regarding  A  or  B  is  available 
due  ’  r>  familiarity  with  the  given  problem  or  one  quite  similar  to  it 
(as  in  sensitivity  analysis).  Factor  (a)  determines  the  iteration  time, 
and  (b)  the  number  of  iterations. 

When  f(x)  is  quadratic,  it  was  observed  that,  due  to  linearity,  (=S) 
can  be  solved  very  rapidly  for  a  sequence  of  trial  sets  by  making  use  of 
partitioning  and  bordering  methods  for  matrix  inversion  and  reinversion. 

In  other  cases,  methods  in  numerical  analysis  for  solving  (partly 
nonlinear)  systems  of  equations  can  be  brought  to  bear,  with  simplifi¬ 
cations  arising  for  special  classes  of  problems  (such  as  linearly  separable 
objective  functions  or  non-negativity  constraints).  As  an  alternative 
to  the  direct  solution  of  (=S)  ,  one  may  wish  to  use  the  fact  that 
xD  is  characterized  ty  the  property  that  it  maximizes  f(x)  subject  to 

C 

a^x  +  b^  =  0  ,  isC  .  If  x'  is  found  by  a  gradient-directed 

S  g  1.  ^ 

search  or  some  other  method,  then  u  is  easily  found  as  -Vf(x  )  Ap(A_  AI)~ 

b  O  b 

where  A  is  a  matrix  with  rows  a.  ,  i  e  S  (partitioning  methods  con 
b  1 

be  used  to  keep  track  of  the  required  inverse  efficiently). 


With  regard  to  factor  (b)  it  must  be  uoserved  that  when  d(S°) 
is  large  not  only  might  a  very  large  number  of  trials  be  necessary,  but 
also  the  amount  of  storage  required  for  computer  implementation  of  the 
rules  of  section  2.2  could  become  onerous.  In  this  case  other  strategies 
for  determining  the  sequence  of  trial  sets  should  be  considered.  For 
example,  if  a  large  proportion  of  all  alarms  ve re  real  (as  has  been 
observed  in  numerous  trial  _-:conples)  tten  the  following  "Mr.rlcov" 
strategy  would  prove  to  be  an  effective  alternative  to  the  one  of  section 
2.2:  heed  one  alarm  at  random.  Storage  is  negligible,  and  although  it 
is  possible  for  this  strategy  to  consume  an  infinite  number  of  trials,  the 
expected  number  of  trials  before  convergence  would  be  substantially 
smaller  than  for  the  previous  strategy  (for  whatever  consolation  it  may 
afford,  we  point  out  that  it  can  be  shown  using  the  theory  of  Markov 
processes  that  the  "Markov"  strategy  leads  to  convergence  within  a 
finite  number  of  trials  with  probability  one). 

In  closing,  we  observe  that  the-  algorithm  can  easily  be  modified  to 

k/ 

incorporate  consistent  and  independent  linear  equality  constraints-' 
as  well  as  inequality  constraints  in  a  more  efficient  manner  than  by 
rewriting  them  as  pairs  of  inequalities  (or  using  some  similar  device). 

It  is  easy  to  see  that  the  indices  of  these  constraints  should  be  included 
in  every  trial  set.  This  may  be  accomplished  by  including  them  in  the 
initial  trial  set  and  then  ignoring  throughout  the  calculations  any 


It  can  be  shown  that  the  hypothesis  that  f  assumes  its  unconstrained 
maximum  can  be  replaced  in  this  case  by  the  weaker  hypothesis  that  f 
assumes  its  maximum  over  the  linear  manifold  determined  by  the  linear 


equality  constraints 
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optimality  alarms  which  they  happen  to  give.  If  the  equality  constraints 
are  not  independent,  a  maximal  independent  subset  thereof  should  be  used. 
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In  all  of  the  propositions  of  this  appendix,  f(x)  is  assumed  to  he 
differentiable,  strictly  concave,  and  to  achieve  its  unconstrained  maximum. 

Lemma  1: 

Ihe  maximum  of  f(x)  is  uniquely  achieved  over  every  non-empty  closed 
convex  subset  of  n-dimensional  Euclidean  space  En  . 

Proof  (outline):  Let  X  be  a  non-empty  closed  convex  set  in  En  .  Put 
Z  =  {z:  z  s  f(x)  for  some  x  e  X)  .  It  is  easy  to  see  that  Z  is  non-empty, 
convex,  and  bounded  from  above;  it  can  also  be  shown  to  be  closed.  The 
attainment  of  the  maximum  follows  immediately,  and  is  unique  by  the  strict 
concavity  of  f  . 

Proposition  1: 

If  SCH  is  consistent  and  independent,  then  the  equations  (=S) 

s  s 

have  a  unique  solution  (x  ,  u  )  • 

Proof:  Since  S  is  consistent,  the  set  {x:  a^x  +  bi  *  0,  i  c  S]  is 
non-empty  as  well  as  closed  and  convex.  By  Lemma  1  the  maximum  of  f  over 

s  s 

this  set  is  achieved  uniquely.  It  i6  well-known  that  a  solution  (x  ,  u  ) 
of  (=s)  is  a  necessary  and  sufficient  condition  for  this  maximum. 

s  s 

Hence  x  is  unique.  By  the  independence  of  S  ,  u  is  also  unique. 
Proposition  2: 

Assume  that  (P)  is  feasible.  Ihen  there  exists  a  triple 

#  #  #  # 

(x  ,  u  ,  S  )  satisfying  (KT-1, ...,  U)  ,  and  x  is  unique.  If,  in 

* 

addition,  B  is  independent,  then  u  is  also  unique. 


23 


Proof:  By  Lemma  1,  (p)  has  a  unique  optimum  x  .  The  existence  of 

#  *  *  * 

u  and  S  CM  such  that  (x  ,  u  ,  S  )  satisfies  (KT-1,...,  4) 

is  assured  by  the  necessity  of  the  Kuhn-Tucker  Theorem;  the  uniqueness 

# 

of  x  in  such  a  triple  follows  from  the  sufficiency  of  the  Kuhn-Tucker 

Theorem  and  the  uniqueness  of  the  optimal  solution  of  (p)  . 

# 

When  B  is  independent,  the  uniqueness  of  u  follows  from 
(KT-l)  upon  using  the  uniqueness  of  x*  ,  the  independence  of  B  , 
and  the  fact  that  u*  /  0  only  if  i  e  B  (which  follows  from  the 
uniqueness  of  x  and  (KT-2))  . 

Proposition  3: 

Assume  that  (P)  is  feasible  and  that  B  is  independent.  Let  S 

s 

be  consistent  and  independent  but  not  valid.  Then  either  a^x  +  K  <  0 

g 

for  some  i  e  A-S  ,  or  u  <  0  for  some  i  e  S-B  ,  or  possibly  both. 
Proof:  We  proceed  by  contradiction.  Suppose  that  (i)  a^x  +  K  ^  0  , 

Q 

i  e  A-S  ,  and  (ii)  ut  ^  0  ,  i  c  S-B  . 

Then  from  (ii),  the  Kuhn-Tucker  Theorem,  and  the  definition  of 

s  s  s 

(x  ,  u  )  ,  it  follows  that  x  is  an  optimal  solution  of  the  problem 

Maximize  f(x)  subject  to 

a^x  +  b^  =  0  ,  i  e  SHB 

a^  +  b.  s  0  ,  i  e  S-B  . 
i  l  7 

g 

By  (i),  it  follows  that  x  is  also  optimal  in  the  problem 


2b 

Maximize  f(x)  subject  to 

(1)  a,x  +  \>i  =  0  ,  i  c  SITB 

a  x  +  a  0  ,  i  c  (S-B)  I!  (A-S)  . 

* 

Now  by  the  Kuhn-Tucker  Theorem,  x  is  optimal  in 

(2)  Maximize  f(x)  subject  to  a^  +  bj  i  0  ,  i  c  A  . 

Since  the  feasible  region  of  (l)  is  included  in  that  of  (2),  and  since 

#  # 

x  is  feasible  in  (l),  x  is  also  optimal  in  (l).  But  (l)  has  a  unique 

S  * 

optimal  solution.  Consequently,  x  =  x  . 

S  *  S  * 

To  prove  u  =  u  ,  note  that  x  =  x  implies  S  c  B  and  that 

#  s  #  # 

(x  ,  u  )  satisfies  (=S)  ;  that  (x  ,  u  )  satisfies  (=B)  ;  and  apply 
the  independence  of  B  . 

Thus  (x  ,  u  )  is  identical  with  (x  ,  u  )  ,  and  therefore  satisfies 
(KT-3)  and  (KT-4)  as  veil  as  (KT-l)  and  (KT-2)  .  Consequently,  we 
obtain  the  contradiction  that  S  is  valid. 

Remark:  Proposition  3  implies  that  when  (P)  is  feasible  and  B  is 
independent,  heeding  the  alarms  due  to  a  consistent  and  independent  set 
one  at  a  time  leads  to  at  least  one  set  which  is  one  unit  of  distance 
closer  to  the  closest  valid  set--and,  in  fact,  one  unit  of  distance  closer 
to  every  valid  set. 

Lemma  2: 

Let  1^,  Ig,  and  be  disjoint  subsets  of  M  .  Exactly  one  of  the 

following  systems  has  c  solution: 

(i)  a^  +  b±  =  0  ,  i  e  Ix 

+  b1  s  0  ,  i  e  I2 

ajX  +  ^  a  0  ,  i  e 
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(U)  IjU^UIj  yl  °i  ' 0 

yi  bi  +  1  =  0 

y±  *  0  >  i  e  I2 
yi  2  0  ,  i  e  I3 

Proof :  The  proof  of  this  lemma  follows  fra.*  the  dualty  theorem  of  linear 
programming  [5]. 

System  (i)  has  a  solution  iff  the  linear  programming  problem 
(l)  Minimize  ox  subject  to  system  (i) 

X 

is  feasible.  System  (ii)  has  a  solution  iff  the  linear  programming  problem 


(2) 


Maximize  T,  y,  (-b,  )  subject  to 
y  II  ^  I3  1  1 

I1UI2IJI3  yi  2i  =  ° 


yi 


M  0  ,  i  e  Ig 
l  2  0  ,  i  e  I3 


has  an  unbounded  optimal  value;  for  if  y°  satisfies  (ii)  then  9y°  is 

U' 

0  -•  «  ,  and  conversely  if  the  value  of  (2)  can  be  made  arbitrarily  large 
then  by  the  homogeneity  of  the  constraints  it  can  be  made  equal  to  unity. 
Observing  that  (l)  and  (2)  are  dual  linear  programming  problems  and  that 
(2)  is  feasible  (e.g.  put  y  =  0  ),  by  the  dualty  theorem  we  have  that  (l) 
is  infeasible  iff  (2)  has  on  unbounded  optimal  value.  This  completes  the 
proof , 


feasible  in  (2)  for  all  real  0  £  0  and  _ 


2 ux  ^  <  C’V 


h  9 


as 
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Proposition  U: 

5 

Let  S  c  M  be  consistent  and  independent,  and  let  a  x  ♦  b  <  0  . 

io  1o 

Then  SUiQ  is  inconsistent  if  and  only  if  it  is  dependent,  and  if  it  is 
dependent  then 

w  i h  zi  ai  *  \  ■ 0 

g 

has  a  unique  solution  z  and  {Sljio  -  1}  is  consistent  and  independent 

C 

for  any  i  e  S  such  that  z^^  <  0  .  If,  in  addition,  (P)  is  feasible, 
then  inconsistency  of  SUio  Implies  that  the  excess  of  S  1b  non-empty 

g 

and  that  z^  <  0  for  some  i  therein. 

Proof;  Since  S  is  independent,  SUiQ  is  dependent  iff  (l)  has  a  solution. 

By  Lemma  2,  SUi  is  inconsistent  iff 
o 

ilk  yi  -i  +  yi  ai  »  o 

(2)  °  ° 

ilk  yi  ki  +  \  V  '  -1 

O  O 

has  a  solution.  Thus  to  show  that  SUi  is  inconsistent  iff  it  i6  dependent 

o 

it  is  equivalent  to  show  that  (l)  has  a  solution  iff  (2)  does.  To  establish 
that  (l)  has  a  solution  when  (2)  does,  let  y'  be  a  solution  of  (2)  and 

g 

note  that  it  follows  from  the  definition  of  x  and  Lemma  2  that  y'  >  0  ; 

S  1q 

pence  z^  =  (yj/y^  )  ,  i  e  S  ,  solves  (l).  To  establish  the  converse,  we 

o 

shall  show  that  y'  *  9z^  ,  i  e  S  ,  and  yj  =0  solves  (2)  for  some  real 

o 

0  >  0  if  z'  solves  (l).  Suppose  the  contrary.  Then  it  must  be  that 

<3)  iik  *i  "i  ♦  \  *  °  > 

0 

g 

and  upon  multiplying  (l)  by  x  and  adding  the  result  to  (3)  one  would  obtain 


£7 


i is  *i  (ai  x  *  V  +  (ai  x“  +  bi  >  2  0  > 


which  would  lead  to  a  contradiction  of  the  definition  of  x  .  The  nrcof 
that  SUiQ  is  inconsistent  iff  it  is  dependent  is  complete. 

When  Siji0  is  dependent,  (l)  must  have  a  unique  solution  z  by 
the  independence  of  S  .  To  prove  that  [SUi  -  i}  is  consistent  and 

S 

independent  for  any  i  such  that  z^  <  0  ,  it  suffices  to  prove  this 

C* 

statement  for  any  i  such  that  ^  0  .  Independence  is  an  easy  con¬ 
sequence  of  the  independence  of  S  .  To  shew  consistency,  by  Lemma  2 
it  is  equivalent  to  show  that  the  existence  of  y  such  that 


.  ,  y  a.  +  y.  a.  =0  and  .?>r.  .  y.  b.  +  y.  b.  +1=0,  where 

ieB-i„  i  i  l  i  icG-i^.  l  i  '  "  ; 


o  o 


1o  xo 


z.  f  0  ,  leads  to  a  contradiction.  The  contradiction  is  obtained  os 

follows.  Assume  that  y.  ^  0  ,  for  otherwise  by  Lemma  2  S  -  i„ 

o 

would  be  inconsistent,  which  would  contradict  the  consistency  of  3  . 

Then  Ec  .  (y./y  )  a  +  a.  =  0  .  But  by  the  uniqueness  of  the  solution 

ieo-i^  i  i  1  i 

*  o  o 

of  (l),  this  is  impossible  when  zl  0  . 

Finally  we  nreve  that  if  (F)  is  feasible  and  Si  iiQ  is  Inconsistent, 

C 

then  S  has  a  non-empty  excess  and  z^  <  0  for  at  least  one  i  therein. 

C 

Suppose  to  the  contrary  that  z'^  s  0  ,  i  e  S-B  ,  where  we  do  not  rule  out 
the  possibility  that  S-B  =  (  .  Then  for  some  9°>  0  one  would  have 


V 

ieS 

o 

CD 

o 

o 
Z  . 

1 

a . 

*  0° 

a . 

i 

icG 

9° 

S 

z . 

i 

b, 

l 

+  3° 

b. 

l 

where  9°  z’T  ^  0  ,  i  c  S-B  ,  and  by  Lemma  2  the  system 


a.x+b.  =0,  ic  SHB 

l  l 

a^x  +  b^  ^  0  ,  i  c  {S-B}Ui0 


2C 


would  have  no  solution,  thus  contradicting  the  fact  that 


a.x  +  b.  =  0  ,  i  e  B 
1  i 


a_,x  +  b.  >  0  ,  M-B  . 
i  l 


This  completes  the  nroof. 


Lemma  3: 

Let  d  be  a  given  n-vector.  If  u_^  a_^  =  d  has  a  non-negative 

i 

solution,  then  it  has  a  non-negative  solution  u  such  that 
(i  c  S:  u1  >  0}  is  independent. 

l 

Proof :  See  [5,  p.  50]* 


Proposition  5: 

Assume  that  (P)  is  feasible.  Then  the  collection  A  (j  =  1,...,  k) 

J 

of  all  minimal  valid  sets  for  (P)  is  not  empty,  and  each  A  is  consistent, 

A  J 

independent,  a  subset  of  B  ,  and  u^°  >  0  for  each  i  e  A^  .  Furthermore, 
on  arbitrary  set  S°  is  valid  if  and  only  if  A^q  c  S°  c  B  for  seme 
1  *  jo  £  k  . 

Proof:  By  Proposition  2,  there  is  at  least  one  triple  (x  ,  u  ,  S  ) 
which  satisfies  (KT-1, ...,  t)  ,  If  S  is  not  minimal,  then  from  the 

nature  of  (KT-1,..,,  4)  and  Lemma  3  it  is  clear  that  one  may  delete 

■5f  .  *  . 

constraints  from  S  ami  construct  new  couples  (x  ,  u)  satisfying  the 

associated  (KT-1, ...,  U)  until  a  minimal  valid  set  is  obtained.  Thus 

there  is  at  least  ore  minimal  valid  set. 


Let  A.  be  any  minimal  valid  set.  By  the  definition  of  validity. 


A. 

J 


A 


(at  this  point  we  cannot  assert  the  uniqueness  of  u 


there  exists  u 


29 


**  A  # 

such  that  (u  J,  x  )  satisfies  (KT-1, ...,  4)  associated  with  A  . 

J 

It  follows  immediately  that  A  c  B  ,  and  is  therefore  consistent.  From 

J  “A 

the  nature  of  (KT-1, . ..,  h)  ,  u^  >  0  for  all  i  e  A^  ;  otherwise 

a  proper  subset  of  A  would  be  valid.  The  independence  of  A  follows 

J  J 

by  similar  reasoning  with  the  aid  of  Lenina  3. 

The  criterion  for  the  validity  of  an  arbitrary  subset  of  constraint 
indices  follows  straightforwardly  by  arguments  similar  to  those  used  above. 

Proposition  6: 

Assume  that  (P)  is  feasible.  Let  S  cm  be  consistent  and 

s  # 

independent  but  not  valid.  If  x  p  x  ,  then  for  each  minimal  valid  set 

s  s 

Aj  we  have  either  a^x  +  bi  <  0  for  some  i  e  A^-S  >  or  u^  <  0  for 

S  * 

some  i  e  S-B  ,  or  possibly  both.  If,  on  the  other  hand,  x  =  x  , 
then  (S-A^)  0  0  and  u®  <  0  for  some  i  therein. 

S  ,  * 

Proof:  Assume  x  p  x  .  Ife  proceed  by  contradiction.  Suppose  for  some 
1  £  J  sk  that  (i)  a  ,xS  +  b  2  0  ,  i  e  A  -  S  ,  and  (ii)  uj  2  0  ,  i  s  S-B  . 
Then  by  arguing  exactly  as  in  the  proof  of  Proposition  3  with  A  in 

s  *  J° 

place  of  A  ,  one  may  derive  the  contradiction  that  x  =  x  . 

S  * 

Now  we  assume  x  =  x  .  Observe  that  S  c  B  ,  and  therefore  since 
S  is  not  valid  we  have  A.^S,j=l, ...,  k.  Clearly  S  -  A  0  0  , 
j  =  1, . k  ,  for  otherwise  S  would  have  to  be  a  proper  subset  of  some 

A  ,  which  would  contradict  the  uniqueness  of  the  positive  representation 

^  #  k 

of  -7f(x  )  in  terms  of  the  a.  ,  i  «  A^  .  A  fortiori,  ^Q^S  -  A^)  0  0  . 

It  remains  to  show  that  u^  <  0  for  some  i  therein.  A  useful  identity  is 

A<s  -  V  "s  -  8  A A) 


9 
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and  for  simplicity  we  write  I  for  B.0,  A.  . 

0=1  J 

V/e  may  assume:  (a)  that  I  /  (  ,  for  otherwise  S  -  I  =  S  and 
hence  the  desired  conclusion  (which  becomes  u^  <  0  for  some  i  e  S  ) 

n  g 

must  obtain  or  else  (xk',  u1')  would  satisfy  (KT-4)  os  well  as 

C 

(KT-1,  2,  3)  and  S  would  be  valid;  and  (b)  that  u^  <  0  for  some 

5 

i  c  I  ,  for  otherwise  the  desired  conclusion  must  obtain  or  else  u^  ^  0 
for  all  i  e  S  and  S  would  again  be  valid.  1/e  shall  prove  that 


u^  <  0  for  some  i  e  G  -  I  by  supposing  the  contrary  and  finding  iQ 
k 

such  that  S  A  -  iQ  is  valid,  thereby  contradicting  the  fact  that 
every  valid  set  contains  at  least  one  A. 


To  find  such  on  i  ,  it 

o 


suffices  to  find  i 


I  :  nd  u  such  that 


(l)  Vf(x*)  +  I  .  u  Q,  =0  fiad  u  s  0  ,  i  e(S  1  A  -i 

iefsA  A.-i  }  1  1  1  J_1  J  ° 

J-l  J  o1 


for  then  (x*,  u)  satisfies  (KT-1,...,  U)  associated  with  A^-i^} 


Ue  now  nroceed  to  find  such  an  i  an 

o 


u 


A 


A 


Nov  for  each  minimal  valid  set,  we  have  u.  >  0  ,  i  c  A.  ,  and 

i  j 


u  satisfies 


A 


7f(x  )  +  .  I  u.J  a.  -  0  . 
'  ica.  l  i 

J 


A  . 


For  convenience,  we  irrite  u  ,  for  u.  .  Partitioning  A.  into  I  and 

ij  i  0 

A.  -  I  ,  adding  all  k  such  equations  and  dividing  by  k  ,  one  obtains 
J 

(2)  7f(x*)  +  i?!  iW]  d  +  X  .  =  T(uU/k)  ai  *  °  • 

i€V 

g 

From  assumption  (b)  and  our  supposition  we  als  have  u^  2  0  ,  i  c  3  -  I  , 

G  G 

and  u‘.  <  0  for  some  i  e  I  ,  and  u^ 

l  ’ 


satisfies 


31 


(3) 


Vf(x  )  +  .  £  ul’  o .  +  .E  u.  a  =  0  . 
'  '  ieS-I  l  l  id  l  i 


Multiplying  (2)  by  (l-t)  and  (3)  by  t  and  adding,  where  t  is  a  real 
number,  we  obtain 


oo 


?f(x*)  ♦  ^  [t  u=  +  (l-t)  jS^u^/k)]  a.  +  t  ^  u;  at 


Put 


(l-t)  u  2  (u  /k)  0=0. 

J  ieA.-I  J 
J 


t  =  Min 

{id:  u1;  <  0} 


s  -  I  jli(ui,A)/( (jli(uijA)  - 

\ 


\ 

S V  1 
-  11  1  ? 


By  construction,  clearly  0  <  tu  <  1  and  [t  u°  +  (l-tu)  E  (u.  *A)]  ^  0 

^  J  *****  J 

for  all  i  c  I  with  strict  equality  holding  for  at  least  one  i  ,  say 
iQ  .  Upon  making  the  appropriate  identifications  of  u  in  (h)  with 
t  =  t°  ,  it  follows  that  (l)  holds.  This  completes  the  proof. 


k 


Proposition  7: 

Assume  that  (p)  is  infeasible.  Let  G  c  M  be  consistent  and 

5 

independent,  and  let  SUi  be  inconsistent,  where  a.  x  +  b.  <  0  . 

°  7  1  1 

o  o 

Then 


(1) 


v  z  a  .  +  a  .  =0 

i 


s  s 

has  a  unique  solution  z  ,  und  z  e  0  if  and  only  if  SI Ji  is  in¬ 
feasible.  Furthermore,  if  SUi  ia  feasible  then  ,!J,  {(SUi  )  -  D.}  /  j 

o  j=i  '  o  yr 

5 

and  z.  <  0  for  seme  i  therein. 

i 


Proof :  The  existence  of  the  unique  solution  z  of  (l)  is  known  from 


32 


C 

Proposition  4,  and  the  assertion  that  z  >0  iff  SUiQ  is  infeasible  is 
a  consequence  of  Lemma  2  and  the  relationship  established  in  the  proof  of 
Proposition  4  between  the  solutions  of  (l)  and  of 


iesSi/i  8i  ■  ° 

t2-)  icsSi/i  *  1  =  0  ' 

Assume  that  SUiQ  is  feasible.  This,  with  the  inconeistency  of 
SUiQ  and  the  definition  of  x  ,  implies  by  Lemma  2  that  (2)  has  a 
solution  y'  such  that  y!  >0  and  y!  <  0  for  some  i  e  S  .  First 


we  show  that  {(SUi  )  -  D  } 

®  J 

Then  SUi  e:  D  for  some 
0  ”  Jo 

SUi  is  feasible.  Since  D 
o 

numbers  w  >  0  ,  i  e  D  , 

1 


i-  t  }  J  =!>•••>  ^  .  Suppose  the  contrary. 

1  s  j  s  d  ,  and  in  fact  S'  !i  c  D .  since 
0  °  Jo 

is  minimally  infeasible,  there  exist  real 
Jo 

such  that 


(3a) 

(3b) 


2  w.  Bl 


ietD  -SUi  1 
Jo 


E  v±  a 
ic{D  -SUI  } 

Jo 


+  E  Wj  ai  =  0 

ie  SUi 

0 

+  E  Vi  b.  +  1  =  0  . 
ie  SI Ji 

o 


Letting  y  =  y'  in  (2)  and  taking  a  convex  combination  of  (2a)  with  (3a), 
and  of  (2b)  with  (3b),  one  obtains  for  any  real  t 


(4a) 

(*b) 


ie[D  -SUi  } 


2  (l-t)  w±  ai  +  2  [(l-t)  wi  +  t  yj]  a.  -  0 

ie  SUi 


2  (l-t)  w  b  +  2  [(l-t)  w  +  t  yn  b.  +1=0 

ie(Dj  -SUio)  1  1  4,  SU1  1  1  1 


Since  y *  <  0  for  some  i  e  S  ,  y '  >  0  ,  and  w  >  0  ,  i  e  D  ,  we  can  choose 

1  xo  1  Jo 

an  appropriate  0  <  t°  <  1  to  make  [(l-t°)  w^  +  t°  yp  ^  0  ,  i  e  SUiQ  , 

with  strict  equality  holding  for  some  i  e  S  .  Hence  a  proper 
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subset  of  D.  would,  by  Lemma  2,  be  infeasible --thus  violating  the 
do 

minimal  infeasibility  of  D.  .  Hence  our  supposition  must  be  wrong  and 
d  °o 

a  fortiori  ^{(SUi^  -  D^}  /  0  ,  J  «  1, . d  .  It  remains  to  show 

C 

that  <  0  for  some  i  therein.  A  useful  identity  is 

j§i«sUi0)  •  V  ■  suio  -  (smo)  Dj  , 

and  for  simplicity  we  write  I  for  (S|.ji  )  A.  D.  . 

®  J  J  - 

Q 

We  may  assume  (a)  that  I  ^  /  ,  for  otherwise  (SL)i0)  -  D ^ }  =  Snio 

3 

and  the  desired  conclusion  follows  from  the  known  fact  that  <  0 

c 

for  seme  i  e  S  ,  and  (b)  that  <  0  for  some  i  e  I  ,  for  otherwise 

3 

the  desired  conclusion  again  follows  from  the  known  fact  that  z^  <  0 

for  some  i  e  S  .  Using  these  assumptions,  we  shall  prove  the  desired 

assertion  by  supposing  the  contrary  and  finding  i#t  I  such  that 
.  d 

(SUi  )  U  D.  -  i„  is  infeasible,  thereby  contradicting  the  fact  that 
o  j  =l  j  * 

every  infeasible  set  contains  at  least  one  Dj  .  To  find  such  an  i#  , 
it  suffices  to  find  i#  e  I  and  y  >  0  such  that 

=  »i  -  ° 

ief(SUio)  Dj-1,} 

f  71  bi  *  1  *  0  ’ 

iet(SUl0)  Dj-1,) 

d 

for  then  by  Lemma  2  {(SUi  )  .0,  D,-i_}  would  be  infeasible.  We  now 

o  j  -l  j  w 

-  S 

proceed  to  find  such  an  i#  and  y  under  the  supposiw-M  that  *  0  , 
i  e  SUiQ  -  I  . 

Since  each  D.  is  minimally  infeasible,  there  exist  real  numbers 
J 

WiJ  >  0  »  1  €  DJ  '  J  =  d  ' 


such  that 


Partitioning  D  into  I  and  D  -  I  ,  adding  all  d  equations  of 
J  J 

type  (5a)  and  dividing  by  d  ,  one  obtains 

ih  jli(uij/a)  *  A  s  <Vd)  ai  -  °  • 

IcDj-I 

Similarly,  the  equations  of  type  (5b)  yield 

<6b>  ih  A<v>  bi  *  A  1  <v«>  bi + 1  •  °  • 

ieDj-1 

Putting  y  =  y'  in  (2)  and  adding  t  times  (2a)  to  (l-t)  times 
(6a)  yields 

Av  /d)J  a  +  E 

1  leSll! 

O 


(7a)  ±ZJ.t  y!  +  (l-t)  .|,(w  /d)]  a  +  Sty! 

lei  l  0  J-  ij  i  ieSUi  -I1 


+  E  E  (l-t)  (v  /d)  a  «  0  . 
3  leD^-I  J 


Similarly,  (2b)  and  (6b)  yield 

d 


(Th)  if,  ct  y;  ♦  (l-t)  3S,(Vd>]  bi  +  us£**i 

o 

+  s  (i-t)  (w  /d)  b 

3  ieD.-I  1J  1 


+  1=0. 


Since  the  signs  of  z i  and  y|  agree  for  i  z  S  ,  by  assumptions  (a) 
and  (b)  and  our  supposition  we  can  choose  an  appropriate  0  <  t°  <  1 
to  make  [t°  yj  +  (l-t°)  j?1(wi^/d)]  i  0  ,  i  s  I  ,  with  strict  equality 
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holding  for  some  i  therein.  Upon  making  the  appropriate  identifications 
in  (7a)  and  (7b),  the  desired  i#  and  y  are  at  hand.  This  completes 
the  proof. 
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Example 

s  # 

The  example  of  Figure  A-l  is  designed  to  show  that  x  =  x  is 
possible  when  S  is  consistent,  indapendent  and  invalid  and  B  is 
dependent.  The  same  conventions  are  followed  here  as  in  the  example  of 
section  2.3  of  the  text. 


Clearly  B  ■  {1,2,3}  ,  and  we  may  take  the  minimal  valid  sets  as 
A1  =  {1,2}  and  Ag  »  {l, 3}  (k  *  2)  .  Thus  the  consistent  and  independent 
trial  set  {2,3}  is  not  valid.  Nevertheless,  x^*^»  x*  . 

In  accordance  with  the  pertinent  assertion  of  Proposition  6,  S  *  {2,3} 
yields  an  optimality  alarm  in  {S  -  A^}  U  {S  -  Ag}  =  {2,3}  ,  namely  for  i  *  3  • 
Heeding  it  leads  to  the  trial  set  {2}  ,  which  yields  a  feasibility  alarm 
for  i  =  1  j  heeding  this  feasibility  alarm  leads  to  {2,1}  ,  which  is 
valid  (i.e.  it  yields  no  alarms). 
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